Real-world performance of point-of-care vs. standard-of-care HIV viral load testing in western Kenya: Secondary analysis of Opt4Kids and Opt4Mamas studies

Routine HIV viral load testing is important for evaluating HIV treatment outcomes, but conventional viral load testing has many barriers including expensive laboratory equipment and lengthy results return times to patients. A point-of-care viral load testing technology, such as GeneXpert HIV-1 quantification assay, could reduce these barriers by decreasing cost and turnaround time, however real-world performance is limited. We conducted a secondary analysis using 900 samples collected from participants in two studies to examine the performance of GeneXpert as point-of-care viral load compared to standard-of-care testing (which was conducted with two centralized laboratories using traditional HIV-1 RNA PCR quantification assays). The two studies, Opt4Kids (n = 704 participants) and Opt4Mamas (n = 820 participants), were conducted in western Kenya from 2019–2021 to evaluate the effectiveness of a combined intervention strategy, which included point-of-care viral load testing. Paired viral load results were compared using four different thresholds for virological non-suppression, namely ≥50, ≥200, ≥400, ≥1000 copies/ml. At a threshold of ≥1000 copies/mL, paired samples collected on the same day: demonstrated sensitivities of 90.0% (95% confidence interval [CI] 68.3, 98.8) and 66.7% (9.4, 99.2), specificities of 98.4% (95.5, 99.7) and 100% (96.5, 100), and percent agreements of 97.7% (94.6, 99.2) and 99.1% (95.0, 100) in Opt4Kids and Opt4Mamas studies, respectively. When lower viral load thresholds were used and the paired samples were collected an increasing number of days apart, sensitivity, specificity, and percent agreement generally decreased. While specificity and percent agreement were uniformly high, sensitivity was lower than expected. Non-specificity of the standard of care testing may have been responsible for the sensitivity values. Nonetheless, our results demonstrate that GeneXpert may be used reliably to monitor HIV treatment in low- and middle- income countries to attain UNAID’s 95-95-95 HIV goals.


Introduction
Kenya has an estimated 1.3 million adults and 139,000 children living with HIV (CLWH).Among those on HIV antiretroviral treatment (ART), approximately 88.4% of adults but only 67.1% of children achieve viral suppression (VS) [1].Routine viral load (VL) testing is needed to monitor HIV treatment and detect treatment failure, to prevent morbidity and mortality, and is recommended by the World Health Organization (WHO), including for low-and middle-income countries (LMICs) [2].Kenya has long supported VL testing and has even increased frequency of VL monitoring among certain vulnerable subpopulations, such as CLWH and pregnant and postpartum women living with HIV [3,4].
In Kenya, VL testing is conducted in centralized laboratories that require experienced laboratory personnel, specialized laboratory equipment, and transport of the specimens to the laboratories.This often leads to long turnaround and results return times to patients and infrequent VL testing due to the time and cost.Point-of-care (POC) VL testing has been shown to decrease both the cost of VL testing in medium and high volume clinics and turnaround times from 7-10 days to less than 1 day which can increase access and frequency of VL testing [5,6].
One POC testing platform is the GeneXpert technology, which was originally developed for tuberculosis diagnostics in 2010 [6].It has since been expanded for PCR amplification of RNA and DNA for many infectious pathogens, including HIV-1.The benefits of the GeneXpert platform for HIV-1 VL testing include a 90-minute runtime, an automated test process, and without a separate PCR room.GeneXpert HIV-1 VL testing has been validated by the company and others [7,8], including in LMICs [6,[9][10][11][12][13][14].
While the GeneXpert testing for HIV-1 VL quantification has been validated for technical performance, there is limited data about its real-world performance.Our group conducted two related studies, Opt4Kids and Opt4Mamas, among CLWH and pregnant and postpartum women with HIV in Kenya using POC VL testing via the GeneXpert platform.Thus, we leveraged already collected data from these studies to analyze real-world performance of GeneXpert HIV VL testing compared to the standard of care (SOC) HIV VL testing.We hypothesized that GeneXpert HIV VL testing would demonstrate high sensitivity, specificity, and percent agreement in detecting lack of VS compared to the SOC HIV VL testing.

Study setting and population
We conducted a secondary, retrospective analysis of data collected in two related studies named Opt4Kids and Opt4Mamas conducted in Kisumu County, Kenya.Kisumu County has an adult HIV prevalence rate of 17.5% compared to Kenya's national average of 4.9% [1].Kisumu's pediatric HIV prevalence is the second highest in the country with an estimated 9000 CLWH in Kisumu County [1,15].

Overview of parent studies Opt4Mamas and Opt4Kids
Opt4Kids study was an open label randomized control trial that explored the impact of POC VL testing, combined with targeted drug resistance mutation (DRM) testing and clinical decision support, on VS among CLWH on ART [16,17].The study enrolled 704 CLWH aged 1-14 in five high-volume treatment centers in Kisumu city and followed them for 12 months from March 2019 to December 2020.Participants were randomly assigned to intervention or SOC groups.The intervention group involved POC VL every 3 months using GeneXpert technology, targeted DRM testing when HIV VL �1000 copies/mL, and clinical support for interpretation of DRM testing results.The control group followed national guidelines with VL testing every 6 months and DRM testing after centralized review by the Regional NyaWest Technical Working Group.
Opt4Mamas study was a prospective cohort study that examined VS rates pre-and postimplementation of POC VL testing in pregnant and postpartum women living with HIV on ART [18].The study recruited 820 pregnant women in the same five facilities as Opt4Kids during their antenatal care visits and followed them for six months postpartum from February 2019 to October 2021.For the pre-intervention group in the study, all the participants were given SOC throughout their care.After the intervention implementation, a new cohort of women was enrolled and underwent POC VL testing every 3 months, targeted DRM testing, and clinical decision support as described above for Opt4Kids.

Sample collection and processing
Samples from Opt4Kids were collected from March 2019 to December 2020 and Opt4Mamas from February 2019 to August 2021.When routine care visits coincided with study visits, study staff attempted to coordinate blood collection from that participant for both routine care and study visit purposes from one venipuncture but with separate phlebotomy tubes.The samples were centrifuged on-site by the local laboratory technicians and who then aliquoted the plasma into vials appropriate for specimen testing.The testing plasma samples were kept at room temperature then sent for the respective SOC and POC VL testing within six hours of sample collection.However, this did not occur in all participants and it was not possible in all cases to discern if samples were collected via the same venipuncture for both SOC and POC VL testing if conducted on the same day.When the study visits could not be coordinated with the routine care visits or when the VL testing schedule differed between the study and routine care timelines, separate blood samples were obtained separately by study or routine care staff and sent for their respective testing.Although multiple samples were collected from the same individual, only one sample per participant was used in this study for analysis.The sample chosen was based on the shortest window of time between the POC and SOC VL results.Sample data were entered and stored in a REDCap project, alongside all study data, and data for this analysis were accessed in September 2022.The lead investigators for the Opt4Kids and Opt4-Mamas studies had access to information that could identify individual participants after data collection though our analytic datasets contained de-identified information only.

HIV-1 RNA quantification
SOC HIV-1 VL quantification was performed by Academic Model Providing Access To Healthcare (AMPATH) Care Laboratory or Kenya Medical Research Institute / Centers for Disease Control and Prevention (KEMRI/CDC) Kisian Laboratory on fresh plasma samples within 24-48 hours of blood collection.Occasionally the laboratories would freeze the plasma samples at -80˚C in order to achieve sufficient batch volumes for testing, but this was not routinely documented.Both laboratories use a combination of automated real-time PCR nucleic acid amplification platforms, including Roche C6800 or C8800, Abbott M2000, and Hologic Panther, to conduct SOC HIV-1 VL testing, and staff were not aware of any POC VL test results.At these laboratories, the samples generally undergo an internal standard operating procedure for certain rejection criteria, e.g., hemolysis, clotting, sample identification mismatch, etc. Internal controls are included in each test run and periodic external quality control and assurance samples are also run.
We performed POC HIV-1 VL quantification via GeneXpert HIV-1 VL quantification assay via routine care laboratory staff, who were not aware of any of the SOC VL test results, at the study sites with study resources (e.g., study purchased cartridges and renumerated laboratory staff) on fresh plasma samples within the same day of sample collection.For the duration of the study, we participated in quarterly external quality control/assurance testing at all participating sites.When indeterminate or invalid results occurred, another vial of the same sample was used to run another POC VL test.

Statistical analysis
We provided descriptive statistics of baseline characteristics of Opt4Kids and Opt4Mamas participant data.Our diagnostic outcome was lack of VS at varying VL cutoffs, ranging from HIV-1 VL �50, 200, 400, and 1000 copies/mL.The outcome was assessed concurrently on samples collected the same day (+/-0 day) for both the POC and SOC VL testing.In secondary analyses, we broadened the time windows to include samples collected within 30 days of one another (i.e., +/-30 days) and 90 days of one another (i.e., +/-90 days), with each broader time window including the prior one.
We visually depicted the concordance of the two test results via scatter plots and reported diagnostic performance measures of sensitivity, specificity, and percent agreement (PA) with 95% confidence intervals (95% CI) for the POC VL test result using the SOC VL test result as the reference or "gold" standard.We generated confidence intervals using Clopper-Pearson method.We cleaned and analyzed the data using R version 4.2.3.We did not calculate any a priori sample sizes.

Ethics review and informed consent
We obtained ethical approval for both studies from the African Medical and Research Foundation (AMREF) and Jaramogi Oginga Odinga Teaching and Referral Hospital (JOOTRH) Institutional Review Boards (IRBs) in Kenya, as well as the University of Washington and the University of Colorado Denver IRBs in the United States (with Opt4Kids registered as a trial, number NCT03820323).For Opt4Kids, study staff at each facility approached potential participants' primary caregivers (usually a parent or guardian) for study participation at routine clinical visits and obtained written informed consent, with additional assent only for participants 13-14 years of age.For Opt4Mamas, study staff used similar approach but only obtained written informed consent from the participant.

Participant characteristics
A total of 506 children and 394 women contributed data for this analysis (Table 1).For the children, at enrollment, the median age was 10 years (IQR 7, 12), their median time on ART was 6 years (3,9), 54.0% were on non-nucleoside reverse transcriptase inhibitor (NNRTI)-containing ART, 67.2% of them at WHO clinical stage of I or II, and 73.5% of them had VS (defined as VL <1000 copies/mL).For the women, at enrollment, the median age was 30 years (26,34), 50.0% had an education level of secondary school or above, median time on ART was 4 years (1, 7), 71.1% were on NNRTI-containing ART, 86.3% were WHO clinical stage I or II, and 77.4% virally suppressed.No adverse events related to VL testing occurred in either study.

Sample characteristics
In total 900 samples were included from participants, 506 samples from Opt4Kids and 394 samples from Opt4Mamas.From the Opt4Kids study, there were 213 samples taken the same day, 345 samples taken within 30 days, and 506 samples taken within 90 days.From the Opt4-Mamas, there were 106 samples taken the same day, 241 samples taken within 30 days, and 394 samples taken within 90 days.

Evaluation of POC VL testing from Opt4Mamas samples
Similarly, to the Opt4Kids samples, overall concordance between the POC and SOC VL test results were qualitatively high (Fig 2 ), and specificity and percent agreement improved with increasing the viral load threshold, while sensitivity was modest for Opt4Mamas samples (Table 3).For samples collected the same day, the sensitivity, specificity, and percent agreement were 66.7% (9.4,99.2), 100% (96.5, 100), and 99.1% (95.0, 100) for VL�1000 copies/mL.Sensitivity varied from 35.3% to 75.0%, specificity varied from 95.5% to 100%, and percent agreement varied from 85.8% to 99.1% for VL thresholds �50 to �400 copies/mL, generally increasing as threshold increased.

Discussion
Our secondary analysis of samples collected for both POC and SOC VL testing among children and pregnant/postpartum women with HIV on treatment in Kenya suggests that the GeneXpert HIV-1 VL quantification platform generally performed well against the "gold standard" during this real-world implementation of POC VL testing.As expected, performance improved as the VL threshold cutoff for viral non-suppression increased.Nonetheless, even with low VL cutoffs, specificity (>90%) was high while percent agreement was moderately high (>70%) between POC and SOC VL testing.Performance was reduced as the number of days between the collection time of the two samples increased.Sensitivity was in the 90% range for the highest VL cutoff, but decreased to as low as the 30% range for the lowest or strictest threshold of VL cutoff.
Validation studies comparing the same POC platform of GeneXpert as we used but done in other countries had specificity and sensitivity generally being greater than 90% and percent agreement greater than 80% (see S1 Table ).Colleagues in Botswana used fresh blood samples that were tested within 4 hours of collection with a VL thresholds of >40-1000 copies/mL reported sensitivities of 98.6 to 99.6% and percent agreement of 91-97% [12].Similarly colleagues in India used frozen blood samples that were tested within 2 days of collection with VL thresholds of �40-500,000 copies/mL, their sensitivities were 97%, specificities 97-100%, and percent agreement 82-100% [13].In a study done in Malawi, where colleagues used a VL threshold of �1000 copies/mL and samples collected within 90 days of each other, the calculated sensitivity, specificity, and percent agreement 92.2%, 91.7%, and 90.9%, respectively [8].A meta-analysis comparing GeneXpert to SOC testing had a pooled sensitivity of 96.5% and specificity of 96.6% at a viral load threshold of 1000 copies/ml [19].Despite minor differences in study design, VL thresholds, blood samples, or other elements, all the studies corroborate high concordance of GeneXpert with the gold standard.
Compared to our specificity and percent agreement values, our sensitivity values were lower than expected when compared to sensitivity values reported in other published validation studies of GeneXpert for POC HIV VL testing [6,8,[12][13][14][19][20][21].Also interestingly, as VL cutoff values decreased from <1000 to <50 copies/mL, noting a trend in national and international guidelines to reduce the cutoff for determining viral suppression [2,3], the sensitivity tended to decrease though specificity remained high.This is a likely acceptable tradeoff as identifying individuals not at risk for viremia more accurately could be considered better than missing individuals who are viremic; of course, additional costs of repeat VLs, counseling sessions, and other healthcare interventions for someone truly suppressed are also not trivial.As can be appreciated in our figures (Figs 1 and 2), many of the discordant values are when the POC VL test result is reported as undetectable or lower than limit of detection while the SOC VL test result has a quantifiable value, some of which fall into clinically and meaningfully different levels.This is interpreted as modest sensitivity values for the POC VL test, a few potential reasons may explain this finding.That the sensitivity decreased largely at low levels of viremia raises the concern that the low sensitivity could be due to cross contamination between samples due to high throughput testing in SOC VL testing settings.In other words, if some samples had undetectable VL but due to contamination with other samples with viremia, they erroneously become falsely positive, which would incorrectly render the POC result of undetectable as falsely negative.In a study done in South Africa and Zambia, the authors reported high specificity values but also lower than expected sensitivity values, and found that there were quality control issues in carrying out VL testing due to sample cross contamination [22].Alternatively, it could also be possible that the SOC assays have higher sensitivity to detect VL at low levels, while the POC assay was indeed less sensitive at lower levels of viremia.Given greater variability in user training, test kit handling and storage, testing environments, and protocol of POC assays, these factors too could lower the sensitivity of the POC assay.Despite these possibilities, we do not have reason to believe the POC VL assay we used has some intrinsic limitation with VL detection at low levels of viremia, as this specific range has been interrogated in dedicated validation studies which show high sensitivity values [12,13,19,21].
Nonetheless, given its overall reliability, GeneXpert HIV-1 VL testing may provide a useful measure of high threshold viral failure (�1000 copies/mL) while decreasing cost and turnaround times [16].The determination of usability of POC tests in the field goes beyond just sensitivity and specificity values, the WHO utilizes the REASSURED framework to evaluate POC tests.The framework criteria includes ease of specimen collection, affordability, userfriendliness, rapid results to return, and deliverability [23].Drawing from our own qualitative work and other studies, as GeneXpert HIV-1 VL testing meets these criteria, we remain optimistic for the role of POC VL testing in LMIC [24][25][26].The WHO recommends making POC VL tests as part of routine clinical care and prioritizing POC VL testing in high risk groups like children and pregnant/postpartum women living with HIV due to the need for prompt results [27].Patients can receive VL testing results within the same day rather than wait for several days, which has significant value to both patients and providers [26].With real-world implementation data of POC VL testing such as ours, patients, providers, and policy makers can have confidence in incorporating POC VL testing into standard HIV treatment guidelines in clinics across Kenya and other LMICs, especially at the higher VL thresholds.At lower VL thresholds, while the specificity remained high, the sensitivity dropped below 50% at times; arguably less harm is done to patients for the possibility of erroneously considering them viremic when they truly are not, though, as noted above, consequential costs do exist for these false positives.Anticipated challenges to widescale POC VL testing include obtaining and maintaining POC VL testing platforms for multiple clinics, training staff to use these platforms, optimizing testing platform network utilization via creating testing hubs, and ensuring quality control throughout multiple sites.Overall, with relatively high specificity and percent agreement, the comparison between the SOC VL testing and POC VL GeneXpert testing shows encouraging results and provides credence for integration of POC VL testing in routine care practices based on our real-world usage of POC VL testing in an LMIC setting.
Though our study utilizes real-world data with highest numbers of samples reported to date, there is one major limitation to this study.Our analysis was not conducted as an explicit validation study but rather as a secondary analysis that leveraged existing data from other studies; thus, there is potential for biases though we have not readily identified any.Ideally, for even a validation study using real-world implementation, there would have been one blood sample collected from the participant with one venipuncture and the specimen being split into two tubes for the two testing methods, and all samples would be from the same day.Nonetheless, our findings with real-world implementation provides additional evidence that the overall performance of POC VL testing is stable even across months, and mostly comparable to SOC.Additionally, that we detected low sensitivity, raising the concerns about potential compromises in the "gold standard" testing, is admittedly unsettling.Future studies, intentionally planned for validation purposes, can institute robust protocols under study purview to reduce such potential limitations.

Conclusion
Real-world implementation of POC VL testing using the GeneXpert HIV-1 VL quantification testing platform, compared to SOC VL testing platforms in Kisumu, Kenya, showed high values for specificity and percent agreement and moderate for sensitivity at a VL threshold of PLOS GLOBAL PUBLIC HEALTH 1000 copies/mL.At lower VL thresholds and wider time windows between sample collection, specificity and percent agreement remained relatively high, but sensitivity decreased considerably.This tradeoff may be acceptable while both POC and SOC VL testing techniques are optimized.The performance of POC testing supports its integration into routine VL monitoring and may contribute to efforts in Kenya and other LMIC to reach the UNAIDS 95-95-95 targets.

Fig 1 .
Fig 1. Scatter plots for POC vs. SOC VL test results by testing windows for Opt4Kids.Abbreviations: POC = point-of-care; SOC = standard-of-care; VL = viral load.The grey box in each plot delineates the range of VL values resulted by the SOC VL test when the POC VL was <40 copies/mL (lower limit of quantification).The red text indicates the number of such instances.The blue text indicates the number of instances where both the POC and SOC VL test results were lower than the limit of quantification.https://doi.org/10.1371/journal.pgph.0003378.g001

Fig 2 .
Fig 2. Scatter plots of POC vs. SOC VL test results by testing windows for Opt4Mamas.Abbreviations: POC = point-of-care; SOC = standard-of-care; VL = viral load.The grey box in each plot delineates the range of VL values resulted by the SOC VL test when the POC VL was <40 copies/mL (lower limit of quantification).The red text indicates the number of such instances.The blue text indicates the number of instances where both the POC and SOC VL test results were lower than the limit of quantification.https://doi.org/10.1371/journal.pgph.0003378.g002